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ABSTRACT 

To satisfy federal requirements, the National Center 
for Education Statistics and the Division of Adult Education and 
Literacy planned a nationally representative household sample survey 
to assess the literacy skills of the adult population of the United 
States, to be conducted by the Educational Testing Service with the 
assistance of Wertat, Inc. This report describes the development of 
the National Adult Literacy Survey framework, the development of the 
background questionnaire, the development of simulation tasks, state 
adult literacy surveys, and participants in the development process. 
Results from the survey v^rill provide policymakers and others with 
information on the condition of literacy in the United States. The 
field test was conducted in 1991, and the main data collection took 
place in 1992 with a sample of 14,900 adults aged 16 years and older 
living in the 50 states and the District of Columbia. Black and 
Hispanic households were oversampled to ensure reliable estimates of 
their literacy proficiencies. Results will describe the literacy 
skills demonstrated by the total adult population as well as by 
adults comprising various sub-groups and will characterize these 
skills in terms of demographic and personal background information. 
Seven tables present information about the samples. An appendix of 
sample tasks contains some of the prose, document, and quantitative 
tasks adults were asked to complete. (SLD) 
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Chapter I 

Introduction 



Historically, as a nation, we have 
placed a high premium on literacy 
skills as they affect both individual well- 
being and society at large. During the 
last century, literacy has taken on 
even greater importance as we have 
moved from predominantly an agrar- 
ian to an industrial society. It was 
during this tronsition that our nation 
came to require that increasing 
numbers of people have a basic set 
of literacy skills and knowledge in order 
to meet changing societal needs. In 
addition, the types and levels of 
literacy skills needed for citizenship 
and individual economic advance- 
ment hove changed over time and 
will continue to change as we 
advance into the twenty-first century. 
In our technologically advancing 
society, both the quantity and types 
of written materials are growing, and 
increasing numbers of citizens are 
expected to use information from 
these materials in new and more 
complex ways. 

Within this context, our nation's 
literacy skills have increased dramati- 
cally in response to new literacy 
requirements and expanded opportu- 
nities for social and economic growth. 
Today, we are a better educated and 
more literate society than at any time 
in our history; however, we find our- 
selves in a period of Imbalance when 
the literacy demands seem to surpass 
the levels of ^kill. While in the post we 
relied primarily on our formal educa- 
tion system to correct any such imbal- 



ance, we now recognize that a 
school-centered strategy can be only 
part of the solution. 

Rapid technological, economic, 
and labor market changes demand 
that we pay increasing attention to 
the skills of those already in the work 
force. It is estimated that almost 
80 percent of the projected work force 
for the year 2000 is olready employed. 
As a result, developing new and more 
effective strategies for increasing the 
literacy skills of both the current and 
future work force is essential if our 
nation is to maintain its standard of 
living and compete successfully in the 
global market. Improved literacy skills 
are equally as important for participa- 
tion in our technological society with 
its formal institutions, complex legal 
system, and large government 
programs. 

At the historic education summit in 
Charlottesville, Virginia, President Bush 
and the nation's governors met to 
establish a set of National Education 
Goals that would guide the country 
into the twenty-first century. As 
adopted and reported by members of 
the Notional Governors' Association in 
1990, one of the six goals states- 

By the year 2000, every adult 
American will be literate and will 
possess the knowledge and skills 
necessary to compete in a global 
economy and exercise the rights 
and responsibilities of citizenship. 



Since the goals were adopted, 
Congress passed the National Literacy 
Act of 1991. The purpose of this Act is 
"to enhance ttie literacy and basic 
skills of adults, to ensure that all 
adults in the United States acquire 
the basic skills necessary to function 
effectively and achieve the greatest 
possible opportunity in their work and 
in their lives, and to strengthen and 
coordinate adult literacy programs." 

Few, if any, deny the important role 
literacy plays in our society or the 
advantages afforded those who have 
acquired and demonstrated high 
levels of proficiency; however, identify- 
ing and measuring just what skills 
people have and need to function 
adequately have proven to be difficult 
tasks. Although a number of reports 
have served to focus attention on 
literacy, we do not know with any 
precision what types and levels of 
literacy skills adults living in the United 
States possess or how these skills are 
distributed across major subgroups of 
interest. It is this lack of precise and 
definitive information that the National 
Adult Llieracy Survey (NALS) seeks to 
address by gathering both back- 
ground and cognitive data relating to 
the literacy skills of the nation's adults. 
The information that will be available 
from the survey is critical to policy- 
makers responsible for targeting 
resources and designing and imple- 
menting appropriate actions with 
respect to literacy. 

This publication serves as an interim 
report of the National Adult Literacy 
Survey and, as such, does not contain 
any results. The purpose of this report is 
to set forth the definition of literacy 
and the framework that guided the 



development of the instruments for the 
NALS. This chapter provides an over- 
view of the survey's purpose, conduct 
and survey samples. Chapter 2 dis- 
cusses the framework of the survey, 
including the issues Involved in measur- 
ing literacy, the definition of literacy, 
and literacy scales that were adopted. 
In chapter 3, the scope of the bock- 
grounc^ questionnaires is presented, 
including the issues that are addressed 
by the quastions. The framev/ork and 
process tor developing the literacy 
simulation tasks are presented in 
chapter 4. F ;)ally, chapter 5 discusses 
the state adult literacy survey option. 



Overview of the National 
Adult Literacy Survey 

Tne Adult Education Amendments 
of 1988 require the U.S. Department of 
Education to submit a report to Con- 
gress on the definition of literacy and 
then to report on the nature and 
extent of literacy among adults in the 
nation. To satisfy these requirements, 
the National Center for Education 
Statistics (NCES) and the Division of 
Adult Education and Literacy planned 
a nationally representative household 
sample survey to assess the literacy 
skills of the adult population of the 
United States. In September 1989, 
NCES awarded a four-year contract 
for that purpose to Educational Testing 
Service (ETS) with a subcontract to 
Westat, Inc. for sampling and field 
operations. 

Results from the National Adult 
Literacy Survey will provide 
policymakers, business and labor 
leaders, educators, researchers, and 



citizens with vital information on the 
condition of literacy in the United 
States thai is not currently available 
from the frequently administered 
school-based surveys. Information from 
the survey will: 

• describe the levels of literacy dem- 
onstrated by the total adult popula- 
tion as weli as by adults comprising 
various subgroups, including those 
targefed as "at risk" 

• characterize the demonstrated 
literacy skills in terms of demo- 
graphic and personal background 
information 

• characterize the work force of the 
country with respect to demon- 
strated li-eracy skills and activities 
reported Dy individuals in various 
occupational categories 

• compare assessment results with 
those from the young adult literacy 
assessment conducted by ETS in 
1985 under a grant from the U.S. 
Department of Education and the 
workplace literacy survey con- 
ducted by ETS under contract to the 
U.S. Department of Labor 

• interpret the findings related to 
infoimaNon-processing skills and 
strategies in a way that can inform 
curriculum decisions pertaining to 
the education and training of adults 

• provide an increased understanding 
of the skills and knowledge associ- 
aied with funcNoning in a techno- 
logical society 

The field test for the survey was 
conducted in the winter and spring of 
1991 and the main data collection 
took place in the winter and spring of 
1992. Both the field test and the main 
survey were administered in one-on- 



one interviews by trained interviewers 
who are experienced in interviewing 
techniques and testing methodologies. 
Each interview lasted approximately 
one hour and consisted of about 
15 minutes of background questions 
and 45 minutes of literacy simulation 
tasks. The results will b'^ released in 
1993 and will be presented in several 
reports tailored to the information 
needs of various audiences. 

Survey Samples 

The survey was conducted with a 
representative sample of 14,900 adults 
aged 16 and older living in households 
in the 50 states and the District of 
Columbia. Black and Hispanic house- 
holds were oversampled to ensure 
reliable estimates of their literacy 
proficiencies In order to enable 
comparisons wi1h 1he young adult 
literacy results, the sample included a 
sufficient number of adults aged 21 
through 25 years. No special design 
modifications were necessary 
because the number of 2 1 - to 25-year 
olds in the sample was expected to be 
within the range of the 1350 to 1,850 
young adulfs needed to provide 
estimates tor comparisons. Finally of 
the total sample, 2700 adults were 
aged 65 and older. 

Inmates of federal and state prisons 
were also included in the survey. A 
supplementary sample of 1 ,000 male 
and female inmates over the age of 16 
were surveyed in order to provide sepa- 
rate estimates of literacy levels for 
the incarcerated population. In addi- 
tion, it will be possible to reduce the 
impact of non-coverage on estimates 
of literacy rates for Black males by 



incorporating the incarcerated 
sample into the national sample. The 
expected sample of inmates was 
drawn from about 80 state and fed- 
eral prisons from across the country, 
with an average of 12 to 15 inmor-ifS 
per facility participating. In total, 
approximately 15,900 adults were 
interviewed in households ar^d prisons 
across the country (see Taole 1.1). 



Table 1.1 Expected $6mple Size 
for the.^Nationol Adlilt^teracy Survey 



Category 

Age 16 to 64 
Age 65 and older 
Incarcerated Persons 
Total 



Estimated 
Sample Size 

12,200 
2,700 
1,000 

15,900 



Summary 

The National Adult Literacy Survey 
was planned and irriplemented in 
response to the Adult Education 
Amendments of 1 988, which call for a 
report on the status of literacy among 
the nation's adults. To this end, aduiTs 
aged 1 6 and older from across the 
country were Interviewed in 1992 in 
order to collect background informa- 
tion and cognitive data on the literacy 
skills of the nation's population. The 
results will describe the literacy skills 
demonstrated by the total adult 
population as well as by adults com- 
prising various sub-groups and will 
characterize those skills in terms 
of demographic and personal back- 
ground information. 
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Chapter 2 

Framework Developmer^t 

The purpose of the Notior^ol Adult spread failure of Army recruits on 
Literacy Survey (NALS) is to report on entrance tests during World War ! 
the nature and extent of the literacy suggested that the self-reports about 
skills demonstrated by the adult popu- literacy such as those collected by the 
lotion residing in households in the Census Bureau were not reliable. 
United States, To meet this goal, the Second, there was a growing interest 
NALS builds on recent research relat- in the potential of standardized testing 
ing to the nature of literacy in our for educational purposes. Educotonal 
society as well as on methodological testing was promoted as a means for 
and technological advancements in diagnosing specific learner strengths 
the areas of assessment and psycho- and deficiencies, for describing par- 
metrics. Thus, it is useful to review the ticular learner achievements, and for 
historical context for the investigation measuring program outcomes.^ These 
of literacy to show h( >w the NALS builds factors combined to focus attention 
on earlier worl< and evolving concep- on what will be discussed as the 
tions of literacy. "traditionol approach" to assessing 

literacy. 



Issues Involved in 
Measuring Literacy 

To gauge early literacy rates, 
historians have had to rely upon such 
indicators as counts of signatures 
taken from legal documents, such as 
wills, marriage licenses, and deeds. A 
deliberate effort to study literacy rates 
began in the mid- 1800s when the 
Census Bureau started gathering 
information on self-reported literacy 
rates. The Bureau counted as illiterate 
those who reported that they could 
not read or write a simple message in 
English or any other language. 

By around 1920, however, two 
factors set the stage for a shift away 
from reliance on self-reported statistics 
toward standardized measures of 
reading/literacy based on demon- 
strated performance. First, the wide- 



The Traditional Approach. Through the 
use of standardized objective tests of 
reading achievement, it was possible 
to estimate percentages of various 
population groups performing at or 
above specified reading grade levels. 
This led to attempts to establish a 
criterion for literacy based on grade- 
level scores on reading tebts. Persons 
scoring at or above a specified level 
were considered to have adequate 
reading skills to perform successfully on 
materials or tasks judged to be of 
comparable grade-level difficulty. 
Those persons who failed to attain the 
specified level were labeled as "illiter- 
ate" or "functionally illiterate" and 



Oscar K Buros, "Ftfty Years m Testing Some Reminis- 
cences. Cniicisms, and Suggestions.' Educotionol 
f?eseofChe/- (July-August 1977) 9- 15 



v/ere presunned 1o lack the necGSsary 
reeding skills to function in our society ' 

The use of grade- level tesi scores as 
an indicator of literacy problems 
among adults has some serious limita- 
tions. Grode^level scores ore typicoL/ 
determined from the average perfor- 
mance of an in-school norming 
sample on multiple choice questions 
covering a particular set of school- 
related reading passages. In contrast, 
the literacy materials that adults 
generally encounter in various every- 
day contexts ore different from the 
materials typically associated with 
school-based standardized tests. As o 
result, performance on these school- 
based tests is often not a gooc predic- 
tor of performance on literacy tasks 
associated with non-school settings/ 

An additional consideration is thot 
questions are typically selected for 
inclusion in a standardized test on the 
basis of item statistics designed to yield 
scores that maximally d'iferentiate 
among individuals. Such a procedure 
con result in reliable and valid tests for 
purposes of ranking and selection, but , 
particularly with adults, it is less useful 
for purposes of instructional piece - 
ment, aiognosis of specific strengths 
and weaknesses, or for the certifica 
tion of specific competencies. This 
limitation in part reflects the fact that 
analyses are rarely, if ever, ijr^dertaken 
to determine specific factors contrib- 
uting to task difficulty. Despite this fad, 
the purposes identified above ore the 
very ones for which standardized 
reading achievement tests hove been 
employed in literacy programs for 
adults. Concerns such as these led 
researchers in the 1970s to move to 
what is called here the "competency- 



based approach" to the assessment of 
odult literacy, 

The Competency-Based Approach. 

During the 197Qs, notional perfor- 
nr^once surveys, such as those con 
ducted by Louis Horris and Associates, 
Educational Testing Service, and the 
National Assessment of Educational 
Progress, attempted to go beyond 
school-related reading tasks by includ- 
ing a range of nate'ials more like 
those that adults typically encounter 
at home, at work, or in their communi- 
ties.'' liie most publicized of these 
notior^ol surveys was the Adult Perfor- 
mance Level (APL) project. In addition 
to reading and writing skills, tlie APL 
project included measures of compu- 
tation, problem solving, rjnd interper- 
sonal skills. The results were reported on 
pcfformance measures as they inter- 
octed with content areas such as 
occupational knowledge, consumer 
economics, health, and law.' 
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I'-'^foctfia! CoiK cpt to( Miirguhilfv litiyrato AdiJlif> 
, t tr ji [,>rn-^ort ( !^ M SO H? , Alfxcindfio VA I Uimon 
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Woshir^qinn 0 ; ' Ih(f No I tonal f^oodiiHj Council 
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!h^odif^^ Abiliiy "> tt-)e Unitod States tor the Notioniji 
Reading Ccnicf Woshingtun. D C Iho Nolionol 
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While surveys such os those cited 
previously made significant advances 
over the traditional scliool-Dased 
measures of reading ochicvemenl , 
they also share some of the same 
limitations and assumptions. In these 
studies, no attempt was made to 
onalyze the tasks with respect to the 
cognitive processes required for sue 
cessful response or 1o determine what 
foctors contributed to task difficulty. 
In addition, with the exception of the 
APL, these surveys employed the 
additive scoring model summing 
across items 1o yield a single score. 
Thus, OS wilh earlier standardized tests, 
these surveys treated literacy as an 
ability distributed along a single cofv 
tinuum. Because the single point 
selected 1o represent the standard of 
literacy voried from survey 1o survey, 
the estimates of ''illiteracy" or "func 
tional literacy" varied widely, ranging 
from abour 13 percent to about 50 
percent. While debate ensued as to 
the accuracy of the estimates of the 
extent of the literacy problem and Ihe 
utility of a single benchmark or c jjt 
point, critics pointed to the varying 
definitions of literacy utilized, the 
different standards selected, and the 
differences among the tasks included 
as explanations for the noncom- 
parability of results. An attempt to 
address these criticisms formed the 
basis for what is described next as the 
"profile approach." 

The Profile Approach. In 1985, the 
National Assessment of Educational 
Progress (NAEP), under a grant to ETS, 
developed and conducted a house- 
hold survey of the literacy skills of 



young adults, aged ?1 to 25 * ilie 
purpose of Ihis assessment was to 
examine the exteni and nature of the 
literocy problem among young aduHs. 
Building on previous work in ossossirK.^ 
literacy, the young adult assessment 
design attempted to extend the 
concept of literacy, to toke into 
account the criticism of earlier surveys, 
and to benefit from advancements in 
educational assessment methods 

From its inception, the young adult 
literacy (YAI ) assessment turiphosi/ed 
the importance of collecting bock 
ground information as well as porfor 
mance data in order to gain o bettr>r 
understanding of the condition of 
literacy omong young adults. A back 
grouna end attitude q jestionnoire 
was developed that collected mfor 
motion on family bockgroufid. {OS[K;)n 
dent characteristics, educational 
experiences, work and community 
experiences, cjnd litorocy practices 
Ihe data collected by the bock 
ground questionnaire provided rich 
descriptive information of the young 
adult population and were useful in 
generating group comparisons and 
relational analyses The tasks ttiat 
yielded the performance data simu- 
lated the kinds of activities thot aduitr> 
normally engage m on a daily basis 
and thai require a broad range of 
literacy skills for successful comipletion 
The simulation nature of the tasks 
meant that they were open-er-ided 
in format. 

Because a major goal of the YAI 
study was to estimate literacy 

*\riU?fK\} S \tHJ(\g Axlijifi, \'utur-*, r. '4: . j! "'-] 

ros*'nQ Service 



proficiencies for the young adult 
populof "^n and for certain subpopula- 
ttons, tinere needed to be as brood a 
coverage of content as possible. In 
extending the range of content 
coverage, it was necessary to employ 
some form of item-sampling design 
because the entire set of tasks was too 
large to administer to any one person. 
This could have resulted in talking only 
about distributions of performance on 
individual hems or about mean perfor- 
mance across tasks responded to by 
different samples of individuals. An 
alternative was to apply some form of 
scaling procedure in order to aggre- 
gate information across sets of tasks so 
that summery statements about group 
distributions could be made. The YAL 
study made use of item response 
theory (IRT)/ a statistical method for 
scaling individual test items for diffi- 
culty in such a way that the item has a 
known probability for being correctly 
completed by an adult performing at 
a given proficiency level. This method 
quantifies the level of difficulty of each 
tosk along a scale as well as the 
performance of individuals along the 
same scale so that the likelihood of 
SI iccess on any item is <^ function of 
the item's characteristics and the 
person's proficiency. 

Unlike previous studies, the YAL study 
provided measures along three scales 
that represented three distinct areas 
of literacy — prose, document, ond 
quantitative — that became the three 
scales for reporting the results. The 
distinctiveness of the three scales was 
supported by factor analyses as well 
as conceptual analyses of the vari- 
ables underlying performance.^ These 
analyses suggested that both material 



and task characteristics varied some- 
what among the three scales. Thus, 
using multiple literacy scales instead 
of one comprehensive scale better 
represented the muififcceted nature 
of literacy. Furthermore, the YAL study 
reported the proportions of the 
population that performed at various 
levels along each scale and portrayed 
the performance profiles of several 
population subgroups. The approach 
taken by the young adult study, 
therefore, viewed literacy not cs a 
single dimension along which a single 
cut point or standard can be selected 
to separate the "literate" from the 
"illiterate/' but rather as a set of com- 
plex informa Non-processing skills that 
go beyond decoding and compre- 
hending school-like prose materials. 

Since the release of the results from 
the YAL survey, its profile approach to 
assessing literacy hos been extended 
to three other literacy studies — the 
U.S. Department of Labor (DOL) 
Literacy Assessment"^ and the Missis- 
sippi'^- and Oregon state assessments. 
The DOL assessment was designed to 
profile the lite''acy proficiencies of 
nationally representative samples of 
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persons applying for the Job Training 
Partnership Act (JTPA) program and 
persons participating in either the 
Employment Se vice (ES) or Unemploy- 
ment Insurance (Ul) programs. This 
assessment used the literacy tasks 
cdministered in the YAL survey as well 
as new literacy tasks that were devel- 
oped to complement the original tasks 
on each of the three literacy scales. 
Mississippi and Oregon each con- 
ducted their own state representative 
survey using the identical item pool 
to that of the DOL assessment. 

Other literacy studies that looked at 
adult literacy from a multidimensional 
perspective have been conducted in 
two countries: the Survey of Literacy 
Skills Used in Daily Activities conducted 
in Canada in 1989/'' and the Survey of 
Australian Adult Literacy.'^ The purpose 
of the Canadian study was to provide 
a direct assessment of the functional 
literacy skills of Canada's adult popu- 
lation, aged 16 to 64, in each official 
language. The results were reported 
on three scales: rr^ding, numeracy, 
and v/riting. In coi itrast to the YAL 
sur\/ey, the Canadian study Jid not 
separate out reading prose irom 
reading documents: the reading scaie 
included items that were based on 
both documents and prose passages, 
with a heavy emphasis on documents. 
For the Canadian study, functional 
literacy was defined more in terms of 
the ability to process document-type 
materials rather than to read narrative 
or exposition. The Australian study 
sought to collect data regarding the 
state of adult literacy in order to inform 
debate about the educational 
programs that are necessary to meet 
the technological and economic 



developments in Australia. This study 
also reported results on three scales — 
prose, document, and quantitative — 
which were essentially modeled after 
the young adult literacy scales. 

Defining Literacy 

In developing the young adult 
literacy assessment, NAEP convened a 
panel of experts who, through a con- 
sensus process, helped to set the 
framework by adopting the following 
definition of literacy: 

Using printed and written informa- 
tion to function In society, to acfileve 
one's goals, and to develop one's 
knowledge and potential. 

This definition characterizes literacy 
by focusing on what adults do with 
printed and written information. It 
rejects an arbitrary standard, such as 
signing one'? name, completing five 
years of schooling, or scoring at the 
eighth grade level on a test of reading 
achievement. In addition, this defini- 
tion goes beyond simply decoding 
and comprehending text and implies 
that the information-processing skills 
that adults use to think about content 
are part of the concept of literacy. 

This definition provided the starting 
point for the deliberations of the NALS 
Literacy Definition Committee, which 
was appointed to advise ETS on the 
development and conduct of the 
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survey. In discussing the definition of 
literacy, the committee decided that 
expressing the literacy proficiencies in 
school-based terms is inappropriate 
and that higher-order thinking skills as 
well as the relevance and the context 
of the literacy tasks are important to 
literacy. In addition, committee mem- 
bers decided that while teamwork 
skills, interpersonal skills, and communi- 
cation skills are important for function- 
ing in various contexis, they are not 
part of literacy as defined and are 
beyond the scope of the NALS. They 
further endorsed the idea that literacy 
involves a multiple set of skills arrayed 
along a continuum, rather than dis- 
crete skills that are context bound and 
that literacy skills are employed across 
a variety of adult contexts that include 
work, home, and community. As a 
result of their discussions, the commit- 
tee decided that to revise the defini- 
tion of literacy would no- rather 
than broaden the cone M of literacy. 
The committee, therefore, unani- 
mously adopted the definition of 
literacy used in the YAL assessment to 
guide the development and conduct 
of the NALS. 

The adoption of the above defini- 
tion of literacy for NALS has had an 
impact on the national discussion 
about literacy, including the National 
Literacy Act of 1991 . As defined in the 
Act, literacy is "an individual's ability to 
read, write, and speak in English and 
compute and solve problems at levels 
of proficiency necessary to function on 
the job and in society, to achieve 
one's goals and to develop one's 
knowledge and potential." It might be 
noted that this definition explicity 
recognizes the computational aspect 



of literacy and implies that there are 
multiple aspects of literacy. 

Extending the 
Literacy Scales 

The YAL assessment reported on the 
literacy skills of young adults in terms of 
three scales representing distinct and 
important aspects of literacy. 

• Prose literacy: the knowledge and 
skills needed to understand and use 
information from texts that include 
editorials, news stories, poems, and 
fiction 

• Document literacy: the knowledge 
and skills required to locate and use 
information contained in materials 
that include job applications, 
payroll forms, transportation sched- 
ules, maps, tables, and graphs 

• Quantitative literacy: the knowl- 
edge and skills required to apply 
arithmetic operations, either alone 
or sequentially, to numbers embed- 
ded in printed materials, such as to 
balance a checkbook, figure out a 
tip, complete an order form, or 
determine the amount of interest 
from a loan advertisement 

This framework of three literacy 
scales became the basis of further 
discussions by the Literacy Definition 
Committee. They agreed that literacy 
should not be measured along a single 
continuum, but rather in terms of the 
three dimensions used in the YAL 
assessment. While the committee 
recognized the t other scales, such as 
a writing scale, might be developed to 
capture other aspects of literacy, they 
realized that the resources avoilahie 
placed a constraint on the possibility 
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of their development. Furthermore, 
using printed and written information 
encompasses writing, and each of the 
three scales includes 1asks that require 
the ability to write. As a result, the 
committee accepted the three 
existing literacy scales as the frame- 
work for the survey. 

fhe committee further recom.- 
mended that new literacy tasks be 
developed 1o enhance the three 
existing scales. They decided that the 
development of the new tasks should 
take into account the following: 

• continued use of open-ended 
simulation tasks rather than multiple- 
choice questions 

• continued emphasis on measuring 
a broad range of information- 
processing skills covering a variety 
of contexts 

• increased emphasis on simulation 
tasks that require brief written 
and/or oral responses 

• increased emphasis on tasks that 
focus on asking the respondent to 
descrioe how he or she would set 
up and solve a problem 

• the use of a simple, four-function 
calculator to solve quantitative 
problems 

In addition to information gathered 
through the administration o^ the 
simulation tasks, demographic and 
personal backgrour^d information is 
important for interpreting and report 
ing the literacy results. Thus, a key 
instrument for the survey is a back- 
ground questionnaire, which along 
with the literacy scales provides the 
means to link NAIS to other literacy 
assessments 



Unking NALS to Other 
Literacy Assessments 

One of the major goals of the 
National Adult Literacy Survey is to 
compare its results to those from other 
large-scale assessments of literacy that 
have been conducted during the past 
few years. These include the YAL 
assessment conducted in 1985 by ETS 
for NAEP and the DOL literacy assess- 
ment conducted by ETS. 

In order foi comparisons to be 
made, a significant number of simula- 
tion tasks used in both the YAL and 
DOL assessments ore included in the 
NALS. The use of a common set of 
exercises in the three assessments 
permits the comparison of demon- 
strated literacy proficiencies on the 
three scales among the various popu- 
lations surveyed. For example, com- 
parisons can be made between the 
literacy skills demonstrated in the YAL 
assessment and those of the current 
cohort of 21 • to 25-year-olds. Further- 
more, how literacy skills develop and 
change over time can be explored by 
examining ^he results for the 28- to 33- 
year-old NALS participants who were 
in the oge range of 21- to 25-years old 
in 1985. Moreover, comparisons can 
be made between adults who ore 
eligible for or participating in JTPA. tS. 
and Ul programs and adults in 
selected subgroups from the NALS. 

In addition to the use of a common 
set of simulation tasks, NALS includes 
background questions that were used 
in both of the prior surveys. The inclu- 
sion of these questions enhances the 
compo'-ability of the NALS data with 
the data collected by both the YAL 
and the DOL assessments. 



Summary 

The National Adult Literacy Survey is 
based on the concept that literacy 
involves a broad range of skills that ore 
used in a variety of adult contexts, It 
measures literacy through the adminis- 
tration of tasks that simulate the 
literacy demands that adults encoun- 
ter in their daily lives. The framework for 
the survey encompasses three distinct 
areas of literacy — prose, document, 
and quantitative — each of which is 
rreosured along a continuum or scale. 
Such a framework presents literacy as 
a set of complex information process- 
ing skills and, thus, represents the 
multifaceted nature of the construct 
of literacy. 
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Chapter 3 



Development of the 
Background Questionnaire 



One of the goals of the National 
Adult Literacy Survey (NALS) is to relate 
the literacy skills of the nation's adults 
to o variety of demographic charac- 
teristics and explanatory variables. To 
accomplish this, the NALS includes the 
administration of a background 
questionnaire as well as literacy {^^rfor- 
mance measures. The questionnaire is 
intended to provide data that will 
characterize the adult population of 
the United States, enhance the under- 
standing of the factors that are related 
to the observed distribution of literacy 
skills of the population, and enable 
comparisons with previous studies. In 
addition, a version of the questionnaire 
was developed for the incarcerated 
adults as some of the questions for the 
population at large are not relevant to 
this small subgroup. 

The Scope of the 

Background 

Questionnaire 

As recommended by the Literacy 
Definition Committee, two goals 
guided the development of the 
questionnaire: 

• to ensure the usefulness of the data 
by addressing issues of concern 
given the national scope of the survey 

• to ensure comparability with the YAL 
survey and the DOL literacy assess- 
ment by including some identical 
questions 
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With these goals in mind, the back- 
ground questionnaire addresses the 
following broad issues: 

• general and language background 

• educational background and 
experiences 

• political and social participation 

• labor force participation 

• literacy activities and collaboration 

• demographic information 

The following sections deal with 
each of these issues and describe in 
detail the specific information gathered. 

Genera! and Language Background. 

By design, the NAlS is a study of English 
literacy proficiency. Projected demo- 
graphic changes, however, point to a 
large and growing population of 
adults with limited English proficiency. 
Experience suggests that little or no 
information from the simulation tasks 
in English will be available for these 
individuals and, thus, they can be 
characterized only from the informa- 
tion collected in the background 
questionnaire. In addition, many of the 
questions included in this category 
were important in characterizing the 
sample of young adults in the young 
adult literocy assessment; and, in fact, 
the age of which English was learned 
was found to be a powerful variable in 
the relational onaiyses for the young 
adults. In order to gather as much 
pertinent information as possible, the 
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questions relating to respondent's 
general end language background 
address the following; 

• country of birth 

• education before conning to the 
United States 

• language(s) spoken by others in the 
home 

• language(s) spoken while growing up 

• language(s) spoken now 

• participation in courses fof English as 
a second language 

• self-evaluation of proficiency in 
English end other languages 

Educational Bockground and 
Experiences. Although we can still find 
"self-educated" individuals, formal 
educatior^ remains among the most 
important factors in the acquisition of 
literacy skills Level of education is 
known to be an important predictor of 
demonstrated performance on the 
prose, document, and quantitative 
literacy scales across racial/ethnic 
groups. The questions addressing 
educational background and experi- 
ences are designed to provide data 
for descriptive and relational analyses 
as well as to address some specific 
issues. The questions collect informa- 
tion on the following: 

• highest grade or level of education 
completed 

• reasons for not completing high 
school 

• high school equivalency 

• current educational aspirations 

• types and duration of training 
received in addition to traditional 
school 

• context, that is, school, home, or 
work, in which literacy activities 
were learned 



• physical, mental, or health condi- 
tions that may affect literacy skills 

Political and Socio! Participation. 

People need to read, write, and cal- 
culate in order to accomplish impor- 
tant tasks not only at work and in 
school, but also at home and in their 
communities, The questions included 
under political and social participation 
will make it possible to explore the 
kinds of free-time activities that adults 
engage in relative to demonstrated 
proficiencies. Use of library services is 
important because libraries promote 
reading and often provide literacy 
programs, In addition, because an 
informed citizenry is essential to 
political participation and printed 
material is an important medium for 
conveying public issues, information 
will be collected on hovy adults 
keep abreast of current events and 
public affairs. The questions in this 
section address the following: 

• sources for obtaining information 
about current atlairs 

• television vievv'ing 

• use of librar\' services 

• voting behavior 

Labor Force Participation. There is 
widespread concern that the literacy 
skills of both our present and future 
work forces are not adequate for 
competing in the current global 
economy or for coping with our rapidly 
evolving technological society. The 
questions relating to laoor force 
participation are based on standard 
labor force concepts widely used in 
economic surveys and will allow a 
variety of labor market activity and 
experience variables to be con- 



structed. Cornbined with the data on 
the demonstrated literacy proficiencies 
of adults, the labor market variables 
can be used to examine associations 
between literacy proficiencies and the 
labor market problems and experi- 
ences of key subgroups. In addition, 
the questions included will make it 
possible to link results to the DOL 
literacy survey. The questions in this 
section address the following: 

• employment status 

• weekly wages or salary 

• weeks of employment for the 
last year 

• annual wages or salary 

• industry and occupation 

Literacy Activities and Collaboration. 

Questions relating to literacy activities 
and collaboration will address several 
important issues. Some of the questions 
will provide information about the 
types of materials — newspapers, 
magazines, books, and brief docu- 
ments — that adults read. Thus, the 
information collected will make it 
possible to investigate the relationship 
between the types of materials read 
and demonstrated literacy proficien- 
cies. Anothei subset of questions asks 
about the frequency of particular 
reading, writing, and mathematics 
activities engaged in for personal use 
as well as for use on the job. By asking 
adults about the types of literacy 
practices they engage in specifically 
for work, analyses can relate on-the- 
job literacy practices to various occu- 
pational categories, education levels, 
and income levels. The issue of col- 
laboration is addressed by questions 
that ask if a person receives assistance 



when engaging i particular literacy 
activities. The questions in this section 
collect information on the foilovying: 

• newspaper, magazine, and book 
reading practices 

• reading, writing, and mathematics 
activities engaged in for personal use 

• reading, writing, and mathematics 
activities engaged in for work 

• assistance received from others with 
particular literacy activities 

Demographic information. The inclu- 
sion of demographic variables will 
make it possible to describe the adult 
population as well as ':o investigate the 
demonstrated literacy proficiencies of 
major subgroups of interest, such as 
racial/ethnic groups, males and 
females, and age groups, including 
those over the age of 64. In addition, 
the data will allow for the investigation 
of such issues as the educational 
experiences of White. Black, and 
Hispanic populations as well as their 
access to literacy related services: the 
educational experiences of different 
generations of adults; and the relation- 
ships of socioeconomic status and 
family background to literacy. The 
demographic information that will be 
collected includes the following: 

• educational attainment of parents 

• marital status 

• number of people in family 
employed full-time and part-time 

• sources of income other than 
employment 

• family and personal income from all 
sources 

• race/ethnicity 

• age 

• gender 



Incarcerated 

Background 

Questionnaire 

Because many of the questions for 
the household population are not 
appropriate for an incarcerated 
population, a more relevant version of 
the background questionnaire incor- 
porating questions from the 1991 
Survey of Inmates of State Correctional 
Facilities was developed. The inmate 
survey is sponsored by the Bureau of 
Justice Statistics of the U.S. Depart- 
ment of Justice. 

Most of the questions in the house- 
hold survey quesTionnoire that deal 
with general and language back- 
ground and with literacy activities and 
collaboration remain in the incarcer- 
ated questionnaire. Many of the 
questions dealing with education, 
however, were either revised or 
replaced with questions from the 1991 
inmate survey. These questions better 
reflect the educational experiences of 
inmates both prior to their incarcera- 
tion and while in prison. The questions 
pertaining to political and social 
participation in the household ques- 
tiorinaire were replaced with questions 
from the 1991 inmate survey dealing 
with current offenses and criminal 
history. Some of the questions in the 
household questionnaire dealing with 
labor force participation were 
replaced with questions about 
inmate's prison work assignments. 
Several questions dealing with family 
income and employment status of 
family members — appropriate for the 
household population, bu1 not for the 
incarcerated ~ were dropped from 
the demographic section of the 



questionnaire. As a result of these 
changes, the questionnaire for the 
incarcerated population addresses 
the following niajor topics: 

• general and language background 

• educational background and 
experiences 

• curren; )ffenses and criminal history 

• prison work assignments and labor 
force participation 

• literacy activities and collaboration 

• demographic information 

Administering the 
Questionnaires 

Each of the background question- 
naires is designed to take about 
15 minutes to administer. To ensure 
standardized administration, the 
questionnaire is read to the respon- 
dent by a trained interviewer. For some 
of the questions, the respondent 
receives a card showing response 
alternatives as the interviewer reads 
them cloud. These procedures for 
administering the questionnaire ser\/e 
to minimize or eliminate the need for 
the respondents to call upon reading 
skills in order to respond to the 
questions. 

Both the household and incarcer- 
ated questionnaires were translated 
into Spanish and odministered by 
bilingual interviewers, Spanish is the 
second most prevalent language in 
this country; the non-English, non- 
Spanish language groups are not 
prevalent enough across the country 
as a whole that other translations 
would be practical for survey work. 
In addition, many Hispanic respon- 
dents may not be able to complete 
the assessment's simulation tast^ in 
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English; thus, it was considered impor- 
tant to collect background informa- 
tion in order to understand particularly 
the language background and 
literacy experiences of that group. 
Since the survey is intended to assess 
only the English literacy skills of the 
population, the simulation tasks were 
not offered in Spanish. 

Summary 

With the information gathered by 
the background questionnaires, it will 
be possible to characterize the dem- 
onstrated literacy skills of the nation's 
adults in terms of demographic and 
personal background information. The 
household questionnaire was modified 
for the incarcerated population, 
particularly to collect information 
about inmates' current offenses and 
criminal history as well as their educa- 
tional experiences while in prison. Both 
the household and the incarcerated 
questionnaires were translated into 
Spanish. These \/arious background 
questionnaires will moke it possible to 
issue reports tailored to various audi- 
ences, including the adult education 
community, the business community, 
and the correctional education 
community. In addition, the results 
from the survey can be compared 
with those from the YAL assessment, 
the DOL literacy survey, and the 
1991 inmate survey, 



Chapter 4 

Development of the 
Simulation Tasl<s 



The National Adult Literacy Survey 
(NALS) measures literacy along three 
scales — prose, document, and 
quantitative. These scales comprise 
literacy tasks that simulate The 
demands that adults encounter when 
they interact with printed materials on 
a daily basis. The tasks that are being 
administered in the NALS include 
about 85 tasks that were administered 
in the YAL assessment and the DOL 
literacy survey and about 80 tasks that 
were developed specificolly for this 
suPv'ey. The administration of a com- 
mon pool of tasks in each of the three 
surve/s allows for the calibration of the 
scales to permit valid comparisons of 
results across their different populations. 

The framework used to develop the 
new tasks reflects research that was 
conducted on the tasks from the YAL 
assessment, particularly with respect to 
the processes and strategies involved 
in completing the tasks. Thus, the newly 
developed tasks serve to refine and 
extend the three existing literacy scales. 
Furthermore, because about two-thirds 
of the original tasks contribute to the 
document scale alone, new tasks weie 
needed to provide a better balance 
of tasks among the three literacy scales. 

This chapter describes the develop- 
ment of the new tasks in terms of the 
framework that is modeled after the 
approach of the YAL survey, In addi- 
tion, the scope of the combined pool 



of tasks — that is, the original tasks plus 
the newly developed tosks — is out- 
lined in terms of this framework. Finally, 
the process of grouping the tasks into 
blocks or sections end then assembling 
these blocks into booklets for odminis 
tration is described 

Organizing Framework 
for Task Development 

In developing the new tasks, one 
goal ^^as to complement the tasks on 
the three literocy scales - - prose, 
document, and Quantitative • devel 
oped for the young adult assessment. 
This means including a diversity of 
stimulus materials and designing tasks 
that represent the broad range of skills 
and processes inherent in the three 
domains of literacy Furthermore, the 
tasks ore designed to assess o wide 
variety of skills that reflect the 
demands that adults encounter in 
occupational, community, and home 
settings — skills that involve reading, 
writing, and computing. Because the 
tasks ore meant to simulate the kinds 
of activities that people engage in 
when they use printed materials, they 
are open-ended. 

The underlying principle for the 
development of the new tasks is that 
demonstrated performance on any 
given task reflects the interactions 
among tne following: 
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• the structure of the stimulus malerlal, 
e.g., exposition, narrative, table, 
graph, nap, or adverlisernent 

• the content represerUed and/or the 
context from which the stimulus is 
drawn, e.g., work, home, community 

• the nature of what the individual is 
asked to do with the material, i.e., 
the purpose for using the material 
that guides the strategies needed to 
complete the task successfully 

As demonstrated by research 
based on the performance of the 
young adult literacy tasks, these 
factors, operating in various combina- 
tions, aftect the difficulty of a task and, 
therefore, its statistical chorocteristics 
and position relative to other tasks 
along a given liteiocy scoie. * 

Materials/Structures 

The s^'mulus materials selected for 
the tasks reflect a variety of structures 
or linguistic formats that adults 
encounter in their daily activities end 
are reproduced in their original format 
Most of the prose materials used In the 
survey are expository — that is, they 
describe, define, or inform — since 
much of the prose that people read is 
expository in nature; however, narra- 
tive and poetry are included as well. 
The expository materials include a 
diversity of linguistic structures, from 
texts that are highly organized both 
topically and visually to those that are 
loosely organized. They also include 
texts of varying lengths, from full-page 
magazine articles to short newspaper 
articles of several paragraphs. 

The document tasks are based on 
a wide variety of document structures 



which are cotegorized as tables, 
charts and graphs, forms, mops, and 
miscellaneous documents. Tables 
include matrix documents in which 
inforrrjotion is arrayed in rows and/or 
columns, such as transportation sched- 
ules and lists or tables of information or 
data. Documents that are categorized 
as charts and graphs include pie 
charts, bar graphs, and line graphs. 
Forms are any documents that require 
information to be filled in, and miscel- 
laneous structures include such materi- 
als as advertisements and coupons. 

ihe quantitative tasks involve 
performing arithmetic operation on 
numbers embedded in print and ore. 
therefore, tied to some kind of stimulus 
material, the aiaterials for quontitotive 
tasks include both prose and docu~ 
ment structures as there are no struc- 
tures that are unique to quai^titotive 
tasks. A majority of these tasks, however, 
ore based on document structures. 

Table 4. 1 indicates the percentages 
ot tasks both the original young 
adult and the new as well as the 
total — that ore based on the various 
categories of structures, While it may 
seem that there is a disproportionate 
number of tables, this particular struc- 
ture comprises a wide range of materi- 
als that present information in matrix 
formats using words, numbers, pictures, 
and symbols. Thus, materials such as 
transportation schedules, menus, 
tables of contents, as well as tables of 
information or data, ore categorized 
as this structure. 
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Table .4. 1. Percentages of Tasks by 
Categories of Structures . 



Percent of Tasks 



structure Original 


Now 


Total 


Exposition 


6 


15 


21 


NarralivG/l^ootry 


1 


5 


6 


Tabies 


23 


10 


33 


Chads and Gropl^s 


4 


6 


10 


Forms 


13 


6 


19 


Mops 


1 


2 


3 


Misceiianeous 


4 


4 


8 



Adult Contexts/Content 

Since adults do not read printed 
materials in a vacuum but read within 
a particular context or for a particular 
purpose, materials are used that 
represent a variety of contexts or 
content. Six uduit context/content 
areas have been identified as follows; 

• home and family: interpersonal 
relationships, personal finance, 
housing, and insurance 

• health and safety: drugs and aico 
hoL disease prevention and treat- 
ment, safety and accident preven- 
tion, first aid, emergencies, and 
staying healthy 

• community and citizenship: commu- 
nity resources and being informed 

• consumer economics: credit and 
banking, savings, advertising, 
making purchases, and maintaining 
personal possessions 

• work: occupations, finding employ- 
ment, finance, and being on the job 

• leisure and recreation: travel, recre- 
ational activities, and restaurants 

With respect to selecting contexts 
and contents, an attempt was made 
to include as broad a range as pos- 
sible as well as to select universally 



relevant contexts and conlents. This 
would ensure that they would not be 
so specialized as to be familiar only to 
certain groups and that any disadvan- 
tages for people with limited back-^ 
ground knowledge would be minimized. 

Table 4,2 presents the percentages 
of tasks — both the original young 
adult and the new plus the total ~ 
that are based on the various 
context/content categories. While it 
may seem that there is on overrepre- 
sentation of maleriols that fall into the 
community and citizenship category, 
this category is very brood and 
includes such materials as news articles 
from newspapers and magazines, 
information from county, state, and 
federal governmental agencies, 
transportation schedules, information 
from schools and colleges, and so on. 

The materials and contexts 
described above define the axes of 
the matrix in Table 4.3. The cells with a 
dot indicate that tasks, new and/or 
young adult, based on that poriiculor 
combination of material and context 
ore included in the assessment. About 
90 percent of the prose stimulus 
materials ore expository in nature, 
OS reflected in the distribution of 
expository materials across all the 



Tabl6 4.2. Percentages of Tasks by 
Categories of Context/Content 



Percent of Tasks 
Context/Content Original New Total 



Home/Family 


7 


7 


14 


Health/Safety 


3 


1 


4 


Community/Citizenship 


12 


20 


32 


Consumer Economics 


11 


5 


16 


Work 


13 


2 


15 


Leisure/Recreation 


6 


13 


19 



20 



c 



context/content areas. For the pur- 
poses of the NALS. readir^g narrative 
and poetry is considered a leisure 
octiv and so none of the other 
contexl /content areas Is represented 
by narrative and poetry. The design for 
the survey did not require that tasks 
cover all possible combinations of 
materials and contents or contexts. 

Processes/Strategies 

After the stimulus materials were 
selected, tasks were developed that 
simulate the way people would use 
the materials and that require different 
strategies for successful task comple- 
tion. Prose tasks were developed that 
involve three strategies for processing 
information: locating, integrating, and 
generating informotion For/ocofe 
tasks, readers must match on informa- 
tion given in the question with either 
literal or synonymous information in the 
text (see tasks 1 1 and 12. page 34). 
Integrate tasks require readers to pull 
together two or more pieces of infor- 
mation located at different points in 
the text. Generate tas/<:5 require read- 
ers not only to process information 
located at dilferent points in the text, 
but also to go beyond that information 



by drawing on their knowledge about 
a subject or by making brood text- 
based inferences in order to produce 
new information (see tasks on pages 
35 and 36). 

Of the original prose tasks, about 
one-third ore locate tasks while the 
remaining two-thirds ore generate 
tasks. Of the new prose tasks devel 
oped for the supv/ey. about two-thirds 
ore locate tasks, one-fourth integrate 
tasks, and the remainder generate 
tasks. Of the total item pool — the 
original and new combined — slightly 
over half the tasks ore locate, just 
under a third ore generate, and the 
remainder are integrate tasks. 

The strategies required by docu 
ment tasks also include locating, 
integrating, and generating informa 
tion OS well as cycling through informa- 
tion. Tor locate tasks, readers must 
match on one feature or cotegcr/ of 
information given in the task with 
either 'don+lcal or synonymous infor- 
mation in a document (see task on 
Social Security cord, page 37). Cycle 
tasks require the reader to repeat the 
matching process by identifying all 
instances that satisfy a set of condi- 
tions stipulated in the question or 
directive (see task on employment 



Table 4.3. Matrix of Coritext by Materials 



CONTEXT/CONTENT 

Home/Family 
Health/Safety 
Community/Citizenship 
Consumer Economics 
Work 

Leisure/Recreation 



Exposition NarroHvo/Poetry Tobl 



MATERIALS 

l«3 ' Charh/Graphs forms K4aps MJic«lkjn«<HJs 



21 



form, page 37). In completing inte- 
grate tasks, readers must either match 
on two or more features located in 
different ports of the document or 
compare and/or contrast information 
(see task on page 39). As with prose 
generate tasks, document generate 
tasks require readers to go beyond 
information in the document either by 
drawing on their knowledge of the 
subject or by making inferences to 
produce new information. 

About two-thirds of the original 
document tasks are locate, with the 
remaining tasks about evenly divided 
among the integrate, generate, and 
cycle categories. Of the new docu- 
ment tasks, about two-thirds are 
locate tasks os well, and about one- 
fourth are integrate tasks, c:nd the 
remainder are generate tasks; there 
are no cycle tasks. Of the total docu- 
ment pool, about two-thirds are locate 
tasks and one-tenth each are gener- 
ate and cycle tasks with the remaining 
tasks categorized as integrate. 

Quantitative tasks require readers 
to perform arithmetic operations — 
addition, subtraction, multiplication, 
or division — either singly or in combi- 
nation. For some tasks, the type of 
operation is obvious from the wording 
of the question or directive (see task 
on page 41), while for other tasks 
readers must infer which operation 
they should perform (see number 4 on 
page 42 and the task on page 43). 
Similarly, sometimes the numbers that 
are required to perform the operation 
ore easily identified (see task on page 
41), while for other tasks the required 
numbers may be embedded in text 
that has distractors or other numbers 
that might seem plausible to use in 



setting up the problem (see task 
number 5, page 42 and the task on 
page 43). In addition, some tasks 
require readers to explain hovs/ they 
would solve a problem rather than just 
producing a numerical answer (see 
task on page 43), and others require 
the ust^ of a simple, four-function 
calculator to solve the problem. The 
representation of numerical informa- 
tion associated with the quantitative 
tasks includes whole numbers, deci- 
mals, percentages, fractions, and time 
(hours and minutes). " 

Of the original quantitative tasks, 
about one-fourth each are catego- 
rized as addition and subtraction. 
About one-third of the tasks involve a 
combination of operations and the 
remaining tasks are evenly divided 
between the operations of multiplica- 
tion and division. The new quantitative 
tasks are fairly evenly dis .ibuted across 
the five types of operations, with there 
being slightly more subtraction and 
slightly fewer division tasks than the 
other three kinds. Across the total 
quantitative pool, about one-fourth 
each are addition, subtraction, and 
combinotion tasks, with about an even 
number of multiplication and division 
tasks making up the remaining tasks. 

The materials and processes 
described above define the axes of 
the matrix in Table 4.4. The cells with a 
dot indicate that tasks with that par- 
ticular combination of material and 
process are included in the pool of 
literacy tasks for the assessment. For 

■ I r-f rnoro dP^O'i and e^omoles of the kinrJs O'ose 
document ano :iunntita!ivo tor.ks see irwm S Ktrsch 
Ann Jungebiut. and Anne Campbell Beyund the 
Schoo' DcKxs Ihe literacy Needs of Job Seekers 
y>c'ved by the U S Deparlmcnt o! Labor Pnnce^on 
'^jj ! ducatio^a' Test-nq Sowre Soplemncf IVP? 
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example, some tasks based on exposi- 
tory materials require subtraction, but 
there ore no expository-based tasks 
requiring addition. The design for the 
survey did not require that tasks cover 
all possible combinations of materials 
and processes. 

Given the strategies for processing 
information that are required, the tasks 
are open-ended rather than multiple 
choice. That is. they require readers to 
engage in activities that are similar to 
those they might perform if they 
actually encountered the materials 
and, thus, are not constrained by an 
artificial set of response requirements. 
Examples include reading and 
responding to editorials, news stories, 
and classified listings in a newspaper; 
writing a letter to a credit department; 
explaining the differences between 
two types of job benefits; completing 
a bank deposit slip; writing a check; 
keeping a running balance in a check 
ledger; and filling in a for . to order 
merchandise from a catalog. 

Because the tasks are open-ended, 
they require a variety of response 
modes. For some tasks, the respon- 



dents are asked to underline or circle 
information in the stimulus or copy 
information from it. For tasks that 
require completing a form, respon- 
dents copy information from the 
directive or question onto the form. 
In some cases, the information to be 
copied involves numbers that are then 
used to perform an arithmetic opera- 
tion. Other tasks require respondents to 
produce an answer, such as making 
inferences based on information in the 
stimulus or explaining how to set up 
and solve a quantitative problem, 
incorporating a variety of response 
modes ensures that the simulation 
tasks reflect real-life uses of printed 
materials. 



Task Difficulty 

Each of the types of tasks described 
above extends over a range of diffi- 
culty on the three scales. The difficulty 
of a particular task is a result of the 
interaction of the type of process or 
strategy required by the task with other 
variables. For the prose and document 
tasks these other variables include: 



Table 4.4. Matrix of Materials by Process 



PROCESS 

Locate 

Integrate 

Generate 

Cycle 

Add 

Subtract 

Multiply 

Divide 

Combination 



MATERIALS 

Exposition Narrative/Poetry Tables Charts/Graphs Forms Maps Miscellaneous 
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• the number of categories or fea- 
tures of information in the directive 
that the reader has to process 

• the number of categories or fea- 
tures of information in the text or 
document that can serve as 
distractors or plausible answers 

• the degree to which the information 
given in the question has less obvi- 
ous identity with the information 
stated in the text or document 

• the length and density of the text 
or the structure of the document 

Two tosks based on a newspaper 
article about a marathon swimmer 
(reprinted in the appendix, page 34) 
are exam.ples of how these variables 
may interact to effect difficulty. The 
first directive requires the reader to 
match "banana and honey sand- 
wiches, hot chocolate, lots of water 
and granola bars" in the third para- 
graph with the word "ate" in the 
directive. There ore no distractors as 
the article contains only one reference 
to food eaten during the sv/im. As a 
result, this task is one of the easiest 
on the prose scale. The second task 
directs the reader to identify the age 
a1 which Ms. Chanin began swimming 
competitively. In this instance, the 
swimmer's current age of 23 appears 
early in the text and serves as a 
distroctor for when she began com- 
peting, which is given later in the 
article as age 15. Although the stimulus 
material is the same and both tasks 
require locating information, the 
presence of the distroctor serves to 
make the second task significantly 
more difficult than the first. 

An analysis of quantitative tasks has 
shown that the informoTir^n processing 



required to complete the tasks affects 
their difficulty. In general, it appears 
that many adults can perform simple 
arithmetic operations when both the 
numbers and the types of operation 
ore mode explicit. The tasks become 
increasingly difficult, however, when 
these some operations ore performed 
on numbers that must be located and 
extracted from different types of texts 
or documents that contain plausible 
but irrelevant numbers, or when these 
operations must be inferred from the 
directive. As a result, the difficulty of 
quantitative tasks seems to be a 
function of: 

• the particular operation called for 

• the number of operations needed 
to perform the task 

• the extent to which the numbers ore 
embedded in printed materials 

• the extent to which on inference 
musi be mode to Identify the type 
of operation to perform 

For example, one of the least 
demanding of Ihe quantitative tasks 
requires the reader to enter and total 
two amounts on a bonk deposit slip 
(see appendix, page 41). For this task, 
both the numbers and the operation 
ore judged to be easily identified, 
and the operation involves the simple 
addition of t^o decimal numbers that 
ore set up in a column format and 
do not require carrying. A significantly 
more difficult task requires the reader 
to compute the cost of a meal and 
then to determine the correct change 
from a specified amount (see appen- 
dix, page 42). Several factors interact 
to affect the difficulty of this task: Ua/o 
operations ore required; the numbers 
needed to compute the cost of the 
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meal ore found in different parts of the 
menu; and an inference must be 
made to identify the types of opera- 
tions to perform. 

Because the NALS is being adminis- 
tered to a nationally representative 
sample, it is important to capture 
the full range of literacy skills that 
people possess and not just to focus 
on those adults who may have low 
level literacy skills. The tasks included in 
the survey, therefore, cover a range of 
difficulty across each of the scales. 
Thus, one goal in developing the new 
literacy tasks was to extend and refine 
the literacy scales as represented by 
the tasks from the young adult literacy 
assessment. The number of tasks on 
each scale, including both the original 
young adult and the new. that are 
judged to be easy, average, and hard 
is shown in Table 4.5. 

Assembling the Tasks 
for Administration 

From a pool of about 1 10 new tasks 
developed for the survey, about 80 
tasks were selected and assembled 
into seven blocks or sections. Each 
block is designed to take about 15 
minutes of administration time. In 
assembling the new blocks, the follow- 
ing factors were taken into account: 



Table 4.5. pistribution of Tasks by 
Difficulty across the Scales* 



Prose Document Quantitative Total 



Easy 


5 


13 


7 


25 


Average 


22 


57 


27 


106 


Hard 


14 


1 1 


9 


34 


Total 


41 


81 


43 


165 



• the inclusion of roughly an equiva- 
lent number of tasks from each of 
the three literacy scales 

• the inclusion of a broad range of 
content from the identified adult 
contexts 

• the inclusion of a wide variety of 
materials and structures 

• a range of difficulty across the tasks 
as determined from field-test data 

• representation of content relating to 
various racial/ethnic groups 

• a variety of response modes 

• the assignment of all the quantita- 
tive tasks requiring the use of a 
calculator to one block 

Of the tasks that were selected for 
the final survey, 27 were selected from 
the prose scale, 25 from the docu- 
ment, and 28 from the quantitative. 
These tasks were distributed as evenly 
as possible across the seven new 
blocks. Because the new item pool 
could in and of itself became the basis 
of a future assessment, it vyas deemed 
mere important to include a balanced 
number of new tasks from each scale 
rather than to achieve balance across 
the entire pool of both original and 
new tasks (see Table 4.5 for the total 
number of tasks by scale — both 
original and new — included in the 
survey). 

With respect to achieving balance 
in content relating to various 
racial/ethnic groups, it is irnportant 
to note thot balance is achieved 
across the entire sot of stimulus materi- 
als used in the survey ~ the ones for 
the newly developed tasks plus the 
original young adult materials — not 
just within one block. About 55 percent 
of the stimulus materials are neutral 



with respect to both gender and 
race/ethnicity — that is. they do not 
contain any references to people 
In the remaining materials, the refer- 
ences to men and women are about 
equal, and references to specific 
racial/ethnic minorily groups are found 
in about 25 percent of the materials. 
In the remaining 75 percent, the 
references are either neutral with 
respect to race/ethnicity or the 
roce/ethnicity of the person referred 
to is identifiable only if someone might 
have background knowledge about 
that particular person. 

In addition to seven blocks of new 
tasks, a core set of six literacy tasks — 
two from each of the three scales — 
was assembled These tasks are rela- 
tively easy and serve as transition from 
the background questionnaire to the 
simulation tasks. The core is designed 
to take five to 10 minutes to complete. 

The full set of 165 tasks ensures 
broad, balanced, and representative 
coverage of materials and content; 
however, it would take about three 
and a half hours for each respondent 
to complete that number of tasks. 
Because about 45 minutes of response 
time was thought to be a maximum 
that respondents could reasonably be 
expected to spend on the literacy 
tasks, some form of item sampling 
procedure was essential The design 
most suitable for this purpose is a 
powerful variant of standard matrix 
sampling called balanced incomplete 
block (BIB) spiralling In BIB spiralling, as 
m standard matrix sampling, no 
respondent is administered all of the 



tasks in the assessment pool. Unlike 
standard matrix sampling, however, 
in which i1ems or tasks are assembled 
into discrete booklets, BIB spiralling 
allows for the estimation of relation- 
ships among all the tasks in the pool 
through the unique linking of blocks. 

With this approoch. the 13 blocks of 
tasks — the seven new blocks and the 
Six young adult and DOL blocks — 
were assembled into 26 assessment 
booklets, each of which contained a 
unique combination of three blocks. 
In addition, each booklet included the 
section of core tasks. The application 
of the BIB design resulted in the con- 
figuration of booklets, as shown in 
Table 4.6. In this design, each block 
appeared with the same frequency — 
in six of the 26 booklets — and each 
block was paired one time with every 
other block Position effects were also 
controlled for at the block level since 
each block appeared twice in each 
of the possible positions in the book- 
lets — first, middle, and last. 

The spiral component of the design 
orders the books for administration so 
that each booklet is completed by a 
random sample of respondents. One 
outcome of the BIB spiral design is that 
every task is taken by a randomly 
equivalent subsample of respondents. 
This ensures that reliable estimates of 
population performance can be 
calculated for e\ery task. An addi- 
tional benefit of this methodology is 
that every pair of tasks is taken by a 
representative subsample of the total 
sample so that correlations between 
pairs of tasks can be estimioted. 



26 



Table 4.6. Balanced Incomplete 
. Block Design for 26 Booklets 





Core 




Blocks 




1 


c 


1 


2 


13 


2 


c 


2 


3 


9 


3 


c 


3 


4 


7 


4 


c 


4 


13 


8 


5 


c 


13 


9 


6 


6 


c 


9 


7 


10 


7 


c 


7 


8 


1 1 


8 


c 


8 


6 


12 


9 


c 


6 


10 


5 


10 


c 


10 


11 


1 


1 ] 


c 


1 1 


12 


2 


12 


c 


12 


5 


3 


13 


c 


5 


1 


4 


14 


c 


1 


3 


8 


15 


c 


2 


4 


6 


16 


c 


3 


13 


10 


17 


c 


4 


9 


1 1 


18 


c 


13 


7 


12 


19 


c 


9 


8 


5 


on 




7 
/ 


O 


1 
I 


21 


c 


8 


10 


2 


22 


c 


6 


11 


3 


23 


c 


10 


12 


4 


24 


c 


1 1 


5 


13 


25 


c 


12 


1 


9 


26 


c 


5 


2 


7 



Summary 

The underlying principle guiding the 
development of the simulation tasks 
is that demonstrated performance 
on any given literacy 1ask is influenced 
by the structure of the text, the context 
from which it is drawn, and the strate- 
gies employed to complete the task. 
Thus, the simulation tasks are meant to 
cover a range of materials, contexts, 
and strategies and require a variety of 
response modes. Criteria for selecting 
the tasks for administration include 
achieving a range of difficulty, a 
representation of content relating to 
various racial/ethnic groups, and a 
balance in the kinds of tasks — that is, 
prose, document, and quantitotive. 
Given the time allotted to the adminis- 
tration of the literacy tasks, balanced 
incomplete block spiralling was 
employed. 
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Chapter 5 



state Adult Literacy Surveys 



The goal of any assessmenl is to 
conduct a study that provides reliable 
and valid information on a representa- 
tive sample of some population — in 
this case, adults over the age of 16 
residing In households in the United 
States. In accomplishing this goal for 
the National Adult Literacy Survey, it 
will be possible to report on the literacy 
skills of the nation; however, not every 
state will be part of the national 
sample and adults sampled within a 
particular state will not necessarily be 
representative of the population of 
that state. 

The state Adult Literacy 
Survey Option 

To provide states with an opportu- 
nity to understand better the literacy 
skills of their particular populations, 
each of the 50 states was invited to 
participate in a concurrent survey that 
will provide results comparable with 
those of the national study. To accom- 
plish +^'S, the suh/ey instruments — that 
is, the background questionnaire and 
simulation tasks — used in the state 
sur^/eys were to be identical to the 
ones used in the national survey, 
and the data were to be collected 
within the same time frame as the 
national. 

To make the opportunity available 
to stotes, a letter of invitation describ- 
ing the nature of Ihe study, the two 
available models and their associated 
costs, and a time line was sent to state 



directors of adult education, state 
directors of JTPA programs, state 
literacy initiative directors, and gover- 
nors. One available model assumed 
that ETS and Westat would be respon- 
sible for all aspects of the literacy 
sun/ey within the state. The other 
model assumed that the state agency 
or its designated unit would take 
responsibility for recruiting and super- 
vising staff to prepare the listing of 
residential addresses for sample 
selection and to conduct the sun/ey 
at selected households. Under both 
models, ETS and Westat would draw 
the sample, provide sun/ey instru- 
ments, conduct training of field staff, 
score and analyze the data, and 
report the results. 



Tine Participating States 

While many states expressed an 
interest in conducting a concurrent 
state literacy survey, 12 states entered 
into a contract with ETS to conduct 
such a survey. These states are: 
California New Jersey 

Florida New York 

Illinois Ohio 
Indiana Pennsylvania 
Iowa Texas 
Louisiana Washington 

All states selected the first model 
whereby ETS and Westat would be 
responsible for all aspects of the 
sun/ey, including recruiting and super- 
vising staff to conduct the survey. 
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The State Samples 

Each state sample includes 
1 ,000 adults aged 16 to 64. While the 
national sample includes adults cv^-t 
the age of 64, an age limit was placed 
on the state samples because the 
original design for the national sample 
called for such a limitation and the 
age eligibility was expanded after 
state contracts had been negotiated. 

The state sample design is intended 
to provide data that are representa- 
hve of the state as a whole; however, 
the sample of 1 ,000 participants will 
not be large enough to be representa- 
tive of locc ' communities. The state 
data will provide information by 
degrees of urbanization within the 
state. In addition, the data will provide 
precise estimates of literacy proficien- 
cies for single variables of interest, 
such as gender, education level, age 
ranges, and work force participation, 
but the sample size imposes limitations 
on interpretations extended to cross- 
tabulations involving combinations 
of variables. 

As noted above, the instruments 
administered in the state surveys are 
identical to the national survey instru- 
ments and the state and national 
data will be collected during the same 
time frame. As a result, data collected 
in a state as part of the national 
sample will be used to extend and 
supplement tie data from the specific 
state sample. The supplemental data 
from the national somiDle, however, 
may not sufficiently increase the 
representation of subpopulations of 
interest within a particular state. 



The Background 
Questionnaire 

Although the national background 
questionnaire gathers a wealth of 
information about respondents, the 
participating states were given the 
opportunity to include up to five 
additional questions that would give 
them the opportunity to collect 
information pertinent to their particular 
state. The criteria for these questions 
were that they had to be concerned 
with information that is directly related 
to the subject of literacy and they 
could not be rewordings of questions 
in the national questionnaire. These 
questions are included at the back of 
the national questionnaire and are 
asked of all respondents in a partici- 
pating state, regardless of whether 
they are in the national or state 
sample. Thus, the background data 
can be fully supplemented by the 
national sample as described above. 

Reporting the 
State Results 

As a result of their participation in 
the concurrent state literacy survey, 
the 12 states will be able to compere 
their data directly with the national 
data. Each participating state will 
receive a report discussing the demon- 
strated performance of adults aged 
16 to 64 on each of the three literacy 
scales — prose, document, and 
quantitative — using one-way analysis 
by gender, major racial/ethnic groups, 
age groups, and other variables, such 
as educational attainment, current 
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employnnent status, and language 
background. Furthermore, a computer 
tape containing complete responses 
to the background questions and 
simulation tasks as well as a data 
compendium will enable states to 
conduct further analyses. Overall, the 
states will hove availoble important 
baseline information that will aid them 
in planning literacy programs and in 
gauging progress in addressing the 
literacy needs of their populations. 



Summary 

Twelve states conducted a concur- 
rent literacy survey that will provide 
reliable and volid estimates of the 
literacy skills of adults aged 16 to 64 
residing in their states, tt will also 
enable the stotes to compare their 
results with those of the national 
survey. In order to occomplish Ihese 
ends, the background questionnaire 
and simulation tosks administered in 
the state survey were identicol to 
those for the national survey, and they 
were administered during the some 
time period. Reports on the state 
surveys will be issued following the 
reports on the Notional Adult Literacy 
Survey in 1993. 
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Appendix 



Sample Tasks 

This appendix contains sample passages and items classified by scale, struc - 
ture, and process. In addition, tt^e scoring guide is provided for each task. Ar^ 
asterisk next to a score point indicates tt^e correct answer. These questions ore for 
illustrative purposes only and in no way represent the full range of tasks included 
in the assessment. 

Prose Tasks 

Material/structure: Exposition Process/Strategy: Locate 

Find the itticle "Swlnuscr completes MiniuctAn nurathoQ" oa ptge 2 
the newrpApcr provided tad ancwtr the following quortkitu. j 

1 L Uaderline the icntcncc thAt tells whst Ms. Chinifi ste duriDg the 
swim. 

12. At what ife did Cbsnin bc|;ia iwimming coaipetitivcly7 | 



Swimmer completes 
Manhattan marathon 

Th* At»«ci*fd IV«M haUinbtrortindttninMrorthe 

NEW YORK-Univertity of niw Tut by iwtmmltjg .boul 

Mirylind wnlor Sttcy Chmln 28.4 mil« • w«k. Tht Yonktrt 

on W*dnMdiy b*c«mt tht nm nilivt hii comp«t«l •# i twlm- 

Ptwm to twim thw« 20.mit« mtr linct iht wit 15 tnd ^optd 

iKpt •round Mtnhitun. ^ ptriu«dt Olympic •uthoritiM 

Chinin. 53. of Vlriinit. toiddtloof-dtiuwiwlmmmt 

climbed out of thf Gtit Riv«r tt c^n^' 

%th Str#et nt 930 |).m. She Th« UuktmU Socltty of 

bt|inthtftwim>tnounonTui«* Amtrica lolicUtd pl*d|H (or 

d»y. e«ch milt tht iw«m. 

Aipoktimmfofthttwimmer, In July 1M3, Julii RIdct be. 

Roy Brunttt. ukJ Chinin hid c«mt thf Tint per»on to iwim 

kept tjp her tttenith with Around M«nhitUn twicfl. With 

"hinini and honey" sand* htr thrtt Up*. Chmm cimit up 

wichii, hot chocolftlt. lou of jutt thcri of Diaiti Nynd't dis- 

wBltr and (itnoli btn," tinct rtcord.Ht on ■ Flotida-to* 

Chinm Hm twict circird Mm- Cuba iwim. 



Scoring guide for "Swimmer completes Manhiattan marathion" 

11. ' 1 : Underlines, circles, or puts a mark next to the sentence beginning A 

spokesman for the swimmer or underlines, circles, or puts a mark nexl 
to any part of the sentence that just lists the foods. 

2: Underlines, circles, or puts a mark next to any other sentence. 

9: I don't know. 

0: No response 

12. ' 1 : 15. Accept underlining or circiing age or the sentence containing the 

age in the article. 
2: Anything other than 15. 
9: I don't know. 
0: No response (blank) 
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Prose Tasks 

Material/structure: Exposition 



Process/Strategy: Generate 



8. What aigumem is Tom Wicker makiiig in hi»^ 



Did U.S. know Korean jet was astray? 



THK COMPUCm" with r»- 
emmenl Into which thi prwi Hm 
iunk line* VltUiim tod Witin 
hii Mldam b*«n mof« vi*- 
Ibli thin on (ho tint innlvinuy 
ct Soviit diiUuction v4 Komn 
Air Urm HithC OOT. 

On Stp<. 1, hMdIinM, of 
cmm. rapdrtcd tht ad- 
ininMration'i i(«UmtnU tUt 
th« evint hbd booalMi. dutinf 
the U.S. ilAnding l» thv 
world rtlstlvt to th«t ot ttw 
U.8.8.R. 

But thv pTfw iifrKttwiy If 
twttd An «uthoriutlv« ftrticit In 
'Hi* Nation (for Aul. iA-Sft) m- 
tibllihinK to a n»a«Miabli c*r- 
(ainty thai num*rmii U-S. |«v. 
tmn«nt M»nelM kniw or ihouJd 
hiv# k-iown, ilmort from Uti 
tnomtnt Might 007 lift Awbof • 
tC«« Alanka. that It waa o^f 
coum and hMdni r<nr Intruilon 
into GoviH ilr *p«c«, abAv* tomi 
of th« moit Mniltivi Soviit nltl* 
Ury initallitlofu. 

Yat no aiincy, milltAry or d- 
villan. wini«d Kliitht 007 or uitd 
to iruldt it out of dinjiM; naithtr 
did tht Jap*n*M. Aa laU m Auf . 



20. in a hri*f!n|. a Sut« Dt)Mrt' 
mtnt ipokeJtmin clilrotd "no 
•l«iry of th» U.S. rsvtmmtnt 
•v«n luMw (ht plUM wu off 
MkUtM and wu in difficulty until 
alUr it waa ibot down." 

It thit'ii Inia, thi lutluH- of 
Tht. Nition'i artlcli— David 
PMrwn, an tuthority on tha D#- 
r#nM Dtpartment'i World WIdi 
MiliUTy Comnand and Control 
S^Uip, who iptnt a y«ir ra- 
MiTchlnx hU Iriiffthy arlicla^ 
concludN. "the fl*bontt« and 
compUx lyiiam of InlalMtvnc*, 
wamln|« and aacurity thit thi 
U.S. hiA built up uv«r dectdct 
•uffertd an unpraccdenKd and 
mind-boffllni oreikdown." 

BvU I'tarson ihoM In *<♦ 
cnJciatJ&c dtUll why il'i tnott 
unlikely lh«rt waa any luch 
"timulUnaoua failur* of Inda- 
pfrtdant Intclllitttc* ■y«<<»nia" of 
tht Navy, Aimy, Alf Korea. Nt- 
(lontl Security Aftncy. CtntMl 
tnUlllffnca Ajtincy "or th« 
Japanm Mlf-dafcnM a<ency"~-> 
all of which, ha ihowa. had abili- 
ty U> Uack niiht 0O7 at virioua 
tUiaa acraai tha Pacifk. 



Tom 
Wlek.r 



What'a lh« alt«rnatlv« to th« 
■tarring Idea of luch a break* 
down? Thit all theae Kffunciei 
d«llb«rat«Iy chuM to fuld« 
the alrlln«r bvck on « lafe 
coura«, becauaa iLa proiact«d 
ovtrftlght of ths Kamchatka 
PenlnauU Mtd SakhnJIn latand 
itould nctlvata Soviet radftfand 
air dnhntet and thui yield « 
'bontntu' of intdllfenc* infor- 
matlon to watching «nd Uatftn* 
tn( U.S. alcctronlc davlcca. 
Dcaptic «t) adminiatralion pro- 
tMia 14 Ova contrary, lha cvl- 
dinc* PMnon preaanta ralMa 
thlt itttm«tiv« It IcBit to th« 
hlKh protMbitity l«v«t. 

But PcMraon does noi aiacrt 
u • fact that iht United SUtef . 
South Kort« or t>othdelib«rataly 
plinnad «d InUlliftncf miulun 
fcr PtlfHt 007; h« cmimAm tht 



poaiibillty that It almply "blun- 
dtftd" fttlo aeMitivt Soviet air 
•pica, and th« tlacLronlc on- 
took».ii for tha United State* d*. 
cldtd on th* ipot to taka In* 
Ulllfenc* •dvantag* of ih« 
error—nevir dreamlni the 
RuaaiiM :nuld ahoel down an 
uurmttf airlloar. 

But If tha diuetar happened 
that way. i'auton nom. two »■ 
pfri«nc«d piloti (nearly 20.COO 
flylni )tour« between than) not 
only nrdi an error in Httinc the 
automatic pilot liut "Ml iit ih^Ir 
cockpit foe live kuuj*. facinf t>M 
autopilot MiKtM iwlteh dlrKliy 
in fitwitof tham at eya l»\il,yet 
failad U) M* that it wae nH im- 
prooetly." Nor In all that time 
could they havs u»*d th« avail- 
able radar and Olhtr eytUtm to 
chMk coum and poiitlcn. 

pMMon aleo praaante eiibetfln - 
tla'( evid«nc« tnat Soviet redar 
detKtion arrd cominunicationt 
•ystema ovar Kamchatka and 
Sakhalin war* being iammad 
that nifhl, which wt>ulil help ac- 
count for their documenud dtf- 
fWty in CAUbiiig Ui Flifkt 



OC?. H« nconctructa electronic 
evIdeiK* too. to ahow that the 
airliner chanc«d coune iKghtly 
afln paaalng near • U.S. n(MS& 
rtcormaieaanM plane; othenMiae 
it would hav* ctoaaMi SaUutin 
far north of the point wWa a 
Soviet figlitar fWwIly iM it 
down. 

The janmlnf afid roum* 
change, aa detallad by Feanoc. 
ktmmly auggaet what ha ob< 
s-iDutly fear*: "that K.A.L 00?'e 
Intruilon inlo So>t«t einpace. 
far from being aa-idental, waa 
Mil oKhaatratad" wit^ tka 
Kcatan administration, at aomt 
Icvtl, doing th* orch**trating. 
Evan if »ot, th* delibaraU 
ailenct-or ahocking fail un -of 
to nany U.S. d*t*ctien ayatema 
argue that Prc«id«nt Raagan and 
th* Mcurity «eUblt«hin*nl hav* 
gnMUr rMponaibility for fll|hi 
WTa fata than th*y *dnll>-or 
that a complaiaant praaa Km 
b**n willing to a9*k. 



Ctifiyt<t*>t* i>M h- TH* Sfw y«f* 



Scoring guide for "Did U.S. know Korean jet was astray?" 

8. ' 1 : Argument identified. Respondents make a statement of the author's 
main point. 

2: Argument not identified. Respondents tell what the article is about 
Others may list evidence without stating what the argument is Or they 
may use the prompt as a basis for personal digression. 

9: I don't know. 



0: No response (blank) 



Prose Tasks 

Material/structure: Poetry 



Process/Strategy: Generate 



11. What is the poet trying to express in this poesi? 



Tbs pedigree of honey 
Docs not concern the Bee — 
A clover, any tinie, to him 
IsAristncracy — 



(Smily Dickinson) 



Reduced Uom cngmai copy 



Scoring guide for Dickinson poenn 

11. 1 No or unsatisfactory interpretation: Respondent makes an unsatisfactory 
or incomplete interpretation of the poem or provides a single word 
response or copies from the poem. 

2: Literal interpretation: Respondent states a plausible summary of the 
poem or explains what the poem is about. 

•3: Thematic interpretation: Respondent states a plausible theme of the 
poem. May also include other information, such as what the poem is 
about. 

9: I don't know. 

0: No response (blank) 
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Document Tasks 

Material/structure: Form 



Process/Strategy: Locate 



301«02-0304 



£M£M£STAW.lSHfOyon 



Scoring guide for Social Security cord 

1. "1: Signs name or marks an X on 
the line. 

2: Any otiier response. 

9: I don't know. 

0: No response (blank) 



Reduced Uom ongm.; copy 



Document Tasks 



Material/structure: Form 



Process/Strategy: Cycle 



1. You hire gone to an employment center (or help in findins » 
tob. You know that thlt center h«adle« miur diilcrent kinds of 
)ob<. Also, Kveta} cf your friends who have Ap^icd bett tuvc 
found iobs that appeal to you. 

The agent has tAk«n your uune and address And fivcn you 
the reft of the form to fill out. Complete the (ofW «o the 
employnent center can help you g«t a )6k. 



BUth dAtc. 



Height- 



Wright. 



3cx: Male FenuJe_ 

Health 



Las« grade completed in school- 
Kind of work wanted: 

Pan-time 

Full-time 



Summer 

Yciff-fouad- 



Reduced trom onginol coDv 



Scoring guide for employment 
application 

la. Personal Information: birth dote, 
age. sex, height, veight, health, 
grode 

* 1 : Satisfactory completion of 
form portion. 

?; UnsaHsfactory completion of 
form portion. 

9; I don't know. 

0: No response (both sections 
totally blank) 



b. Job information: kind of work 

•1: Satisfactory completion of 
form portion, that is, at leas! 
one check in each column. 

2: Unsatisfactory completion of 
form portion. 

9: I don't know. 

0: No response (both sections 
totally blank) 
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Document Tasks 

Material/structure: Table 



Process/Strategy: Locate 



Hexe is a wage and tax sutement that cornea with a paycheck, 

8. What is the cunent net pay? 

9. What is the gross pay for this year to date?. 







03/15/85 




OVIKTlWi 


CKOfS 




Nil »AY 










CU*MNT 


6i5O0 


1 


62500 


1 

) 




50k) 


1 

\ 


I 
1 


so'o 


VCAtt TO DATE 






<i26865 


1 







Suction? 


CK USlON 




UHITCO FO 


nits iHS 




cool 


J 
I 


1 

1 
1 

.J 


\ 
1 

1 i 1 


1 

j 


1 
1 









STATE WH 


CiTYWm 


fICA 


T 


1 

13^5 


1 ■ 

1 

I 


38|31 


YEAR TO 
OAT J 




8250 


j 


261{67 



NON-NEGOTIABLE 



OTHtBPCDuCTlONS 



07 



DEN 



» 



Qedij' eci from (>riainai copy 



Scoring guide for wage and tax statement 



8. -1: 


459.88. 


2: 


Any other amount. 


9: 


i don't know. 


0: 


No response (blank) 


9. '1: 


4268.85 


2: 


Any other amount. 


9: 


1 don't know. 


0: 


No response (blank) 



Document Tasks 



Material/structure: Graph Process/Strategy: Integrate 



13. You arc a maikctiag manager for a small m a fiuf a ct uring tinxL 
This graph shows your company's sales over the last three years. 
Given the seasonal pattem shown on the grsph, pitdia the sales 
for Spring 198S (in thousands) by putting an 'V on the graph* 



1982 1983 1984 1985 




Reduced from ongmof copy 



Scoring guide for sales grapti 

13. • 1 : Puts an "x" or other mark on the graph at any point above the point for 
Winter, 1984, in the area under 1985. 

2: Puts an ^^x" on the graph at any point at or under the point for Winter, 
1984, in the area under 1985 or puts an "x" anywhere under one of the 
other years. 

9: 1 don't know. 

0: No response (blank) 



BEST COPY AVAILABLE 
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Document Tasks 

Material/structure: Map 



Process/Strategy: Cycle 



6. Use this map to follow dizectioQS. 



Begin at the Fourth Avenue side of City Hall. 
Go west to Beech Street. 
Go noxthl block. 
Go west 2 blocks. 

You are now at the comer of 



Office 



Bsnk 



.and 



Fifth Ave. 



Fourth Are. 



•C3 




Second Ave. 



First Ave. 



RQduCQd from original copy 

Scoring guide for map 

6. * 1 : Fifth (Avenue) and Walnut (Street). Accept mark on nnap at correct 
point. 

2: Any other answer. 

9: I don't know. 

0: No response (blank) 
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Quantitative Tasks 

Material/structure: Form 



Process/Strategy: Addition 



8. You wish to deposit a $300 check and $57.23 in cash in a checking 
account* Fill out your deposit slip to do so. List both deposm 
and indicate the total amount deposited. Date yout deposit slip 
May 22, 1985. 



NATIONAL BANK 



(PtmmPrintI 



PltMt «!• your ptnoAiiM d tci rt t ttetcctt. 
!f you nM mort, tt* your ptnei«l btntitr. 





D«M«n 


Ctnti 


CASH 













Nam«- 



.19- 



ns sunR 

CACHITKMta 

^ROMtRLV 

CNOOMSVO 



Tow IttffW 



TOTAL 



CMCCKS AND OTMCH ITEMS ARC RCCeiVKO FO« OOQSIT SUMECT TO ®' 
UNtFOMM COMMIUCIAL COOC OH ANY Am.lCAaua COLUECTION AONKKMCNT. 



Reduced from original copy 



Scoring guide for deposit slip 

Total 

'1: 357.23 or when enters incorrect amounts for "cash" and/or "checks," 
adds them correctly. 

2: Any other amount. Or fails to fill in, when other parts are attempted. 
0: No response: form totally blank. 



mnwumum 
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Quantitative Tasks 

Material/structure: Table 



Process/Strategy: #4, Combination 
#5, Multiplication 



Suppose you had $3.00 to spend for lunch. 

4. If you order a Lancaster Special sandwich and onion soup, how 
much change would you get back? 

5. How much should you leave for a 10% tip? 



Soups — Made by our Chef Daily 

Onion soup -60 

Sou 1 of the day 60 
Vichysso!S€ in Summer 

Beef burgers, broiled to order; 1 85 
'/4 lb. of the finest Beef available, seasoned to perfection 
and served on a buttered bun 

Wine Cheddar cficcsc burger 1 95 

Blue-cheese burger 1 -95 

Pineapple burger 1 95 

Bacon burger 2.10 

Wine Cheddar-cheese & Bacon burger 2.25 



Scoring guide for lunch menu 

4. M: 450. 

2: Any other amount or indicates two amounts even if one is correct. 

9: I don't know. 

0: No response (blank) 

5. 250 or 260. 

2: Any other amount or indicates two amounts even if one is correct. 

9: I don't know. 

0: No response (blank) 



Sandwiches 



Sliced Turkey — Garnished 
Turkey Salad — Garnished 
Chicken SaU»d — Garnished 
TunaFLsh Salad — Garnished 
Sliced Beef Tongue — Garnished 
Grilled Wine Cheddar-Cheese 
The Lancaster Special 

Corned B«ef. Melted Swiss Cheese. Sauerkraut 
on Seeded Rye . . . Need we say more? 



1 30 
.95 
.95 
95 

1.50 
75 

1 95 



Minimum Check at Lunch 1 00 



Reduced from ongmci copy 
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Quantitative Tasks 

Material/structure: Miscellaneous 



Process/Strategy: Combination 



3. YcmoMd to boKxofwSlO^OOa Bod tlic*dl«Hociie Equity LotuoQ 

TM wooU ooopme the total amowt of latcmt diarpM you 
would per aika« this k)MpJ«a.PlMict^th*iatarricw«rwbeB 
70U «■ ntdjr to b«(ia. 



FIXED RATE • FIXED TERM 

gS;^ 14.25% 

LOANS 



tAMPUS. MONTHLY WAYftMENT tCHCDULE 



$1Q,000 
tM,000 



$1M.77 

4wr.o« 



1M M mWi i 14.>i% AM 



Reduced from origmaf copy 



Scoring guide for honne equity loan 

3. *3: Speaker explains the two 

basic steps in computing the 
total interest charges, i.e., the 
monthly payment ($156.77) 
times the number of pay- 
ments (120) equals the total 
loan payment; the total loan 
payment minus the amount of 
the loan ($10,000) equals the 
total interest charges. 

2: The speaker explains one but 
not both steps in computing 
the total Interest charges or is 
vague about the steps, e.g., 
you need to know how much 
you will pay over 10 years. 

1 : The speaker states something 
other than an explanation of 
computing interest charges, 
or gives an incorrect explana- 
tion, e.g., the loan ($10,000) 
times the interest rate (14.25%) 
equals the total Interest 
charges. 



0: No response; nothing on tape; 
no notes by interviewer, or 
interviewer writes in the 
interview guide to the effect, 
"Respondent can't answer 
question." 

9: I don't know, can't do; refusal. 
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